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Abstract. Fault-tolerant distributed algorithms are central for building 
reliable spatially distributed systems. Unfortunately, the lack of a canoni- 
cal precise framework for fault-tolerant algorithms is an obstacle for both 
verification and deployment. In this paper, we introduce a new domain- 
specific framework to capture the behavior of fault-tolerant distributed 
algorithms in an adequate and precise way. At the center of our frame- 
work is a parameterized system model where control flow automata are 
used for process specification. To account for the specific features and 
properties of fault-tolerant distributed algorithms for message-passing 
systems, our control flow automata are extended to model threshold 
guards as well as the inherent non-determinism stemming from asyn- 
chronous communication, interleavings of steps, and faulty processes. 
We demonstrate the adequacy of our framework in a representative case 
study where we formalize a family of well-known fault-tolerant broadcast- 
ing algorithms under a variety of failure assumptions. Our case study is 
supported by model checking experiments with safety and liveness speci- 
fications for a fixed number of processes. In the experiments, we system- 
atically varied the assumptions on both the resilience condition and the 
failure model. In all cases, our experiments coincided with the theoretical 
results predicted in the distributed algorithms literature. This is giving 
clear evidence for the adequacy of our model. 

In a companion paper |18| . we are addressing the new model checking 
techniques necessary for parametric verification of the distributed algo- 
rithms captured in our framework. 

1 Introduction 

Even formally verified computer systems are subject to power outages, bad elec- 
trical connections, arbitrary bit-flips in memory, etc. A classic approach to ensure 
that a computer system is reliable, and continues to perform its task, even if some 
components fail, is replication. The idea is to have multiple computers instead 
of a single one (that would constitute a single point of failure) , and ensure that 
the replicated computers coordinate, and for instance in the case of replicated 
databases, store the same information. Ensuring that all computers agree on 
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the same information is non-trivial due to several sources of non-determinism, 
namely, faults, uncertain message delays, and asynchronous computation steps. 

To address this important problem, fault-tolerant distributed algorithms were 
introduced many years ago. As they are designed to increase the reliability of 
systems, it is crucial that they are in fact correct, i.e., that they satisfy their spec- 
ifications. Due to the mentioned non-determinism it is however very easy to make 
mistakes in the correctness arguments for fault-tolerant distributed algorithms. 
The combination of criticality and difhculty make fault-tolerant distributed al- 
gorithms a natural candidate for model checking. 

Unfortunately, there are three reasons why model checking fault-tolerant dis- 
tributed algorithms is a complicated task, (i) First, they are mathematically com- 
plex, and inherently contain many sources of non-determinism, (ii) Second, there 
is no canonical model, such that each algorithm comes with different — and usu- 
ally complex — assumptions about the environment, in particular assumptions 
on degrees of concurrency, message delays, and failure models. By failure models 
we mean both, the anticipated behavior of faulty components and a resilience 
condition stating the fraction of faulty components among all components of the 
system, (iii) Finally, distributed algorithms are traditionally described in pseu- 
docode. This approach is problematic because every paper comes with a different 
(alas unspecified) pseudo code language. It is often not clear how a given pseudo 
code is related to the computational model that is provided only in natural lan- 
guage. Other authors from formal methods have also argued that the algorithms 
and proofs in these papers are hard to understand for outsiders |15j . 

All these problems result in three verification research problems for the use 
of fault-tolerant distributed algorithms. Together, they close the methodological 
gap between distributed algorithms, verification, and software engineering: 

Formalization problem. There is no modeling language for fault tolerant dis- 
tributed algorithms, and the pseudocode and hidden assumptions make it 
difficult to understand the semantics. Success in this area crucially depends 
on collaboration of researchers from model checking and from distributed 
algorithms. 

Verification problem. Even in the presence of a precise model, there are many 
open problems in the area of verification. A central challenge is parameter- 
ized model checking, i.e., verification of a given fault-tolerant distributed 
algorithm for all system sizes. Note however that verification without ade- 
quate formalization is pointless, as one can never be sure what actually has 
been verified. 

Deployment problem. How can one transfer a formal model to a real-life 
implementation and ensure its conformance with the (verified) model? 

This paper is devoted to the formalization problem. In a companion pa- 
per [18] , we are developing new parameterized model checking methods for fault 
tolerant distributed algorithms. A central and important goal of our work is 
to initiate a systematic study of distributed algorithms from a verification and 
programming language point of view in a way that does not betray the fun- 
damentals of distributed algorithms. The famous bakery algorithm [20] is the 
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most striking example from the literature where wrong specifications have been 
verified or wrong semantics have been considered. Many papers in formal meth- 
ods have claimed to prove correctness of the bakery algorithm as evidence for 
their practical applicability. From a distributed algorithms perspective, however, 
most of these papers miss that the algorithm is not using the strong assumption 
of atomic registers but requires only safe registers [21]. Although this example 
shows that many issues in distributed algorithms are quite subtle, the distributed 
algorithm literature is often not very explicit about them, making it hard for 
non-experts to extract the correct model. 

Reviewing the distributed algorithms literature, the formalization problem 
can be reduced to two central questions, (i) First, the question how algorithms 
are described, and (ii) second, the question how to capture the vast diversity in 
computational models that describe the environment. 

(i) Algorithm descriptions in the literature are based on pseudo code, whose 
semantics is described in a hand waving manner (if it is described at all): In 
particular, details which are considered not interesting for the current distributed 
algorithm are ignored or only hinted at, e.g., the bookkeeping over the messages 
that have been sent and received so far. 

(ii) The example of the bakery algorithm shows that assumptions on computa- 
tional models are very subtle. The distributed algorithms literature does, how- 
ever, rarely state these assumptions precisely but rather present them in natural 
language. This is very unfortunate as there are quite involved assumptions that 
are usually not considered in model checking. For instance, [10] postulates that 
in each run there is a value <P such that between two steps of a process, ev- 
ery other process takes at most <P steps. Assumptions of this kind are crucial 
for fault-tolerant distributed algorithms because there are impossibility results 
for the classic asynchronous interleaving semantics '12]. In other words, without 
these assumptions, all algorithms violate the specification. Apart from inter- 
leaving of steps, non-trivial assumptions can also be found for message delays, 
behavior of faulty processes, behavior of faulty links, and resilience conditions 
on the fraction of faulty process. 

To address both aspects of the formalization problem, we need a novel frame- 
work which is natural and adequate for the distributed algorithms community, 
but precise enough to facilitate automated verification. The current paper rep- 
resents a first step towards this goal. 

Contributions of the current paper. We introduce a new framework for the spec- 
ification of distributed algorithms. We focus on the important class of threshold- 
guarded fault-tolerant distributed algorithms, which we discuss in detail in Sec- 
tion[2] For this class, we introduce a parameterized modeling framework based on 
control flow automata, which is a notion from software model checking extended 
by non-determinism and threshold guards. Our framework facilitates flexible 
fine-grained and adequate description of distributed algorithms under different 
fault assumptions and resilience conditions. For automatic treatment, we have 
a front-end similar to Promela [T7] . With this formalism we can express, e.g., 
several variants of classic asynchronous broadcasting algorithms [32] under differ- 
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ent fault assumptions. Our framework is only the first step towards considering 
various environments that are different from the asynchronous one, e.g., par- 
tial synchrony or round models. This will allow us to express a wider range of 
fault-tolerant distributed algorithms, e.g., |10l4l5ll4l33j . 

After introducing the framework, we provide a case study and experiments 
that show how to translate distributed algorithms into our framework. We dis- 
cuss the formalization of an important family of fault-tolerant distributed al- 
gorithms, which is well-understood. In our experiments we consider different 
fault models, and systematically validate the adequacy of our modeling frame- 
work. The experiments are made for a bounded number of processes, i.e., the 
model checking is relatively straight-forward, but not scalable to large numbers 
of processes. As we do not need abstractions for this tasks, we avoid simplifica- 
tion artefacts. Our experiments show that our modeling framework is adequate. 
Thus we have a starting point for a serious investigation of parameterized model 
checking and of systematic deployment. 

Organization of the paper. We first discuss the standard pseudocode construct of 
fault-tolerant distributed algorithms — namely threshold guards — in Section [2j 
In Section [Sj we introduce a general system model for fault-tolerant distributed 
algorithms that provides parameterized processes, parameterized system sizes, 
and resilience conditions. Section |4] introduces our novel variant of control flow 
automata, and discusses how they can be composed to derive instances of dis- 
tributed systems based on the model of Section [Sj In Section [Sj we describe 
the translation of our case study algorithm to its corresponding control flow au- 
tomaton. Section |6] presents the outcomes of our model checking experiments, 
and Section [7] relates our approach to other existing approaches for specifying 
concurrent algorithms. 



2 Threshold guarded distributed algorithms 

Processes that execute the instances of a distributed algorithm exchange mes- 
sages, and the state transitions of these processes are predominantly determined 
by the messages received. In addition to the standard execution of actions, which 
are guarded by some predicate on the local state, most basic distributed algo- 
rithms (cf. |23|lj ) just add existentially and/or universally guarded commands 
involving received messages: 

if received <m> it^ received <m> 

from some process from all processes 

then action(m); then action(m); 

(a) existential guard (b) universal guard 

Depending on the content of the message <m>, the function action performs 
a local state transition and possibly sends messages to one or more processes. 
Such constructs can be found, e.g., in (non- fault-tolerant) distributed algorithms 
for constructing spanning trees, flooding, or network synchronization [23) . 
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Understanding and analyzing such distributed algorithms is far from being 
trivial, which is due to the uncertainty that local processes have about the state of 
other processes. After all, real processors execute at different and varying speeds, 
and the end-to-end message delays also vary considerably. Viewed from the global 
perspective, this results in considerable non-dcterminism of the executions of a 
distributed system. 

Another very important additional source of non-determinism arc faults. In 
fact, one of the major benefits of using distributed algorithms is their ability 
to cope with faults. In case of distributed agreement, for example, it is guaran- 
teed that all non-faulty processes compute the same result even if some other 
processes fail. Fault-tolerant distributed algorithms hence typically increase the 
reliability of distributed systems [55] . 

In order to shed some light on the difficulties faced by a distributed algo- 
rithm in the presence of faults, consider Byzantine faults [27], which allow a 
faulty process to behave arbitrarily: Faulty processes may fail to send messages, 
send messages with erroneous values, or even send conflicting information to 
different processes. In addition, faulty processes may even collaborate in order 
to increase their adverse power. In practice, Byzantine faults can be caused by 
power outages, bad electrical connections, arbitrary bit-flips in memory, or even 
unexpected behavior due to intruders who have taken over control of some part 
of the system. 

If one used the construct of Example (a) in the presence of Byzantine faults, 
the (local state of the) receiver process would be corrupted if the received mes- 
sage <m> originates in a faulty process. A faulty process could hence contaminate 
a correct process. On the other hand, if one tried to use the construct of Ex- 
ample (b), a correct process would wait forever (starve) when a faulty process 
omits to send the required message. To overcome those problems, fault-tolerant 
distributed algorithms typically require assumptions on the maximum number 
of faults, and employ suitable thresholds for the number of messages which can 
be expected to be received by correct processes. Assuming that the system con- 
sists of n processes among which at most t may be faulty, threshold guarded 
commands such as the following are typically used by fault-tolerant distributed 
algorithms: 

if received <m> from n-t distinct processes 
then action(m); 

Assuming that thresholds are functions of the parameters n and i, threshold 
guards are a just generalization of quantified guards as given in Examples (a) 
and (b): In the above command, a process waits to receive n — t messages from 
distinct processes. As there are at least n — t correct processes, the guard cannot 
be blocked by faulty processes, which avoids the problems of the construct of 
Example (b). In the distributed algorithms literature, one finds a variety of 
different threshold guarded commands. Another prominent example is t + 1^ 
which ensures that at least one message comes from a non-faulty process. 



5 



However, in the setting of Byzantine fault tolerance, it is important to note 
that the use of threshold guarded commands implicitly rests on the assumption 
that a receiver can distinguish messages from different senders. In practice, this 
can be achieved e.g. by using point-to-point links between processes or by mes- 
sage authentication. What is important here is that Byzantine faulty processes 
are only allowed to exercise control on their own messages and computations, 
but not on the messages sent by other processes and the computation of other 
processes. 

3 Parameterized System Model 

We model distributed algorithms via their parameters, the processes, and the 
communication medium, the latter via shared variables. In Section |4j we will 
introduce a new variant of control flow automata that allows to specify processes 
of fault-tolerant distributed algorithms. We will discuss how message passing 
distributed algorithms (as mentioned in Section [2]) can be expressed in such a 
model in Section [S] 

We shall define the parameters, local variables of the processes, and shared 
variables referring to a single domain D that is totally ordered and has the 
operations addition and subtraction. In this paper we will assume that D = Nq. 
We use the standard notion of models denoted by ^. 

We start with some notation. Let F be a finite set of variables ranging over D. 
We will denote by Z?'^', the set of all |y|-tuples of variable values. In order to 
simplify notation, given s £ D'^', we use the expression s.y, to refer to the value 
of a variable y € Y in vector s. For two vectors of variable values s and s', by 
s =x s' we denote the case where for all x £ X, s.x — s' .x holds. 

The finite set of variables V = U U {sv} U AU F, where the separate sets 
are described below. The finite set 77 is a set of parameter variables that range 
over D, and the resilience condition RC is a predicate over dI^I . In our example, 
n = {n, t, /}, and the resilience condition RC{n,t,f) \sn>3t/\f<t/\t> 
0. Then, we denote the set of admissible parameters by Prc — {p ^ D^^^ \ 
RC{p)}. The variable sv is the status variable that ranges over a finite set SV 
of status values. (For simplicity, we assume that only one status variable is used; 
however, multiple finite domain status variables can be encoded into sv.) The 
finite set A contains variables that range over the domain D. The variable .sv 
and the variables from A are local variables. The finite set F contains the shared 
variables that range over D. 

A process operates on states from the set S — SV x D^^^ x D^^^ x D^^K Each 
process starts its computation in an initial state from a set 5" C S'. A relation 
R C S X S defines transitions from one state to another, with the restriction 
that the values of parameters remain unchanged, i.e., for all {s,t) £ R, s =n t. 
Then, a parameterized process skeleton is a tuple Sk = (S", 5"°, i?). 

We get a process instance by fixing the parameter values p € Z?!^': one can 
restrict the set of process states toS'|p = {se5'|s —n p} as well as the set 
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of transitions to R\p — RD {S\p x S'|p). Then, a process instance is a process 
skeleton Sk|p = (S'|p, S^jp, i?|p) where p is constant. 

For fixed admissible parameters p, a distributed system is modeled as an 
asynchronous parallel composition of identical processes Sk|p. The number of 
processes in this parallel composition depends on the parameters. To formalize 
this, we define the size of a system (the number of processes) using a function 
A'': Pflc ~^ I^- On our example, we will model only non-faulty processes explic- 
itly in our case study, and we will thus use n — f for N{n, t, f) in our case study. 

Finally, given p e Prc, and a parameterized process skeleton Sk = {S, S'", R), 
we can define a system instance as a Kripke structure. Let AP be a set of atomic 
propositions. (The specific atomic propositions and labeling function that we 
will consider in this paper will be introduced in Section [4.1[ ) A system instance 
lnst(p,Sk) is a Kripke structure (S'inst, •S'l^^^, i?inst, AP, Ainst) where: 

— The set of (global) states is S^inst = {(cr[l], . . . , cr[7V(p)]) e (5'|p)^(P^ | Vi,j e 
{1, . . . , A^(p)}, a[i] =ruiT f^ij]}- More informally, a global state cr is a Carte- 
sian product of the state a[i] of each process i, where the values of parameters 
and shared variables are the same at each process. 

— S'|°3t = (S'O)^(P) n ^inst is the set of initial (global) states, where {S°)^^p^ is 
the Cartesian product of initial states of individual processes. 

— A transition (ct, a') from a global state a S S'inst to a global state a' G 6*1,151 
belongs to iJinst iff there is an index i, I < i < N{p), such that: 

(move). The z-th process moves: {(j[i], a'[i]) G R\p. 

(frame). The values of the local variables of the other processes are pre- 
served: for every process index j ^ i, I < j < N{p), it holds that 

— Ainst : "S'inst is a state labehng function. 

The set of global states 5|n5t and the transition relation i?inst are preserved 
under every transposition i ^ j of process indices i and j in {1, ... , 7V(p)}. That 
is, every system lnst(p, Sk) is fully symmetric by construction. 

Temporal Logic. We specify properties of distributed algorithms in formulas of 
temporal logic LTL \ X. An formula of LTL \ X is defined inductively as: 

— a literal p or ^p, where p € AP, or 

— F (p, G iy9, U V', V ■0, and ip A ^, where ip and ip are LTL \ X formulas. 

We use the standard definitions of paths and the semantics of the LTL \ X 
formulas [5]. 

Model checking an instance of a parameterized system. Now, we arrive at the 
formulation of a parameterized model checking problem. Given: 

— a domain D, 

— a parameterized process skeleton Sk = {S,So,R), 

— a resilience condition RC on parameters 11 (generating a set of admissible 
parameters Prc), 
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— parameter values p G Prc, 

— and an LTL \ X formula ip, 

check whether lnst(p,Sk) \= tp. 

4 A Modeling Framework for Distributed Algorithms 

In this section, we adapt the general definitions of the previous section to fault- 
tolerant distributed algorithms. First we introduce atomic propositions that al- 
low us to express typical specifications of distributed algorithms. Then, we define 
our control flow automata (CFA) that are suitable to express threshold guarded 
distributed algorithms as parameterized process skeleton. 

4.1 Quantified Propositions for Distributed Algoritlims 

We write specifications for our parameterized systems in LTL \ X. This contrasts 
the vast majority of work on parameterized model checking where indexed tem- 
poral logics are used |3l7l8lllj . The reason for the use of indexed temporal logics 
is that they allow to express individual process progress, e.g., in dining philoso- 
phers it is required that if a philosopher i is hungry, then i eventually eats. 
Intuitively, dining philosophers requires us to trace indexed processes along a 
computation, e.g., Vi. G (hungryj — )• (FeatingJ). 

In contrast, fault-tolerant distributed algorithms are typically used to achieve 
certain global properties, as consensus (agreeing on a common value), or broad- 
cast (ensuring that all processes deliver the same set of messages). To capture 
these kinds of properties, we have to trace only existentially or universally quan- 
tified properties, e.g., part of the broadcast specification (relay) [32] states that 
if some correct process accepts a message, then all (correct) processes accept the 
message, that is, (G {3i. acceptj) — (F (Vj. accept^)). 

We are therefore considering a temporal logic where the quantification over 
processes is restricted to propositional formulas. We will need two kinds of quan- 
tified propositional formulas. First, we introduce the set APsv that contains 
propositions that capture comparison against some status value Z € SV, i.e., 

[Vi. svi = Z] and [3i. svi = Z] . 

This allows us to express specifications of distributed algorithms. To express 
the mentioned relay property, we identify the status values where a process has 
accepted the message. We may quantify over all processes as we will only model 
those processes explicitely that are restricted in their internal behavior, that is, 
correct or benign faulty processes. More severe faults (e.g., Byzantine faults) are 
modeled via non-determinism. For a detailed discussion see Section [5l 

Second, in order to express comparison of variables ranging over D, we add 
a set of atomic propositions AP d that capture comparison of variables x, y, and 
constant c that all range over £); AF^ consists of propositions of the form 

[3i. Xt + c<yi\. 
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We then define AP to be the disjoint union of APsv and AP o ■ The labehng 
function Ainst of a system instance lnst(p, Sk) maps its state a to expressions p 
from AP as follows: 

[Vi. = Z] e A|„st(CT) iff /\ ia[i\.sv = Z) 

l<i<JV(p) 

[3z. = Z] e A|„st(CT) iff V {a[i].sv = Z) 

l<i<N(p) 

[3i. a;j + c < j/i] e Ainst(cr) iff \/ {a[i\.x + c < a[i].y) 

l<i<N{p) 

4.2 CFA for Threshold Guarded Distributed Algorithms 

Processes that run distributed algorithms execute the same acyclic piece of code 
repeatedly. In the parlance of distributed algorithms, a single execution of this 
code is called a step, and steps of correct processes are considered to be atomic. 
Depending on the actual code, one can classify distributed algorithms by what 
may happen during a stop. For instance, in our case study, a step consists of a 
receive, a computation, and a sending phase. Therefore, we are led to describe 
steps using the concept of control flow automata (CFA), where paths from the 
initial to the final location of the CFA describe one step of the distributed 
algorithm. 

A control flow automata CFA is a link-labeled directed acyclic graph A = 
{Q, qi, qp, E) with a finite set Q of nodes, called the locations, an initial location 
qi G Q, and a final location qp G Q. A path from qj to qp is used to describe 
one step of the distributed algorithm. The edges have the form E C QxOpxQ, 
where Op is the set of operations whose syntax is defined as: 



Vcir ::= (name of a variable from AU F) (1) 

sval ::= (an clement of SV) (2) 

par am :;= (name of a parameter variable from 7J) (3) 

condvar ::= var | s (4) 

lin_f orm ::= param | int | lin_f orm + lin_f orm | lin_f orm — lin_f orm (5) 

threshold ::= lin_f orm (6) 

guard ::= sv = sval | threshold < var | guard A guard | -iguard (7) 

atomcond ::= condvar < condvar + lin_f orm (8) 

cond ::= atomcond | cond A cond (9) 

Op ::= sv := sval | inc var | guard | var := e where cond (10) 



In addition to constructs of standard control flow automata, we use the state- 
ment "e where cond" that non-deterministically chooses a value s that satisfles 
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condition "cond," if such a value exists, otherwise the statement blocks. More- 
over, there is a special variable sv ranging over SV . Most importantly, our thresh- 
old guarded commands can be expressed as combinations of threshold conditions 
via guard. 

Operational semantics. To distinguish the notions of states in a process skeleton 
and states in a CFA, we call states in a CFA valuations while states in process 
skeletons are called states. The set of valuations are defined identically to the set 
of states defined in Section [| as SV x dI^^I x dI^I x dI^L Then, the following 
shows the semantics where we denote by w |= cond[w'.a::/e] that v models cond 
if all occurrences of e in cond are replaced by v' .x: 

{v,v') G [guard] iff v \= guard A v' ~ v (11) 

{v,v') G Isv := sval] iff v' .sv = sval A v =v\{sv} ^' (12) 

{v. v') £ [[ inc x\ iff v'.x — v.x + \ A v —v\{x} v' (13) 

{v,v') G [a; e where cond] iff v |= cond.[v' .x / e\ A v —v\{x} (14) 

In Section [5] we discuss how one can obtain a CFA from a description of 
a distributed algorithm based on pseudo code used in the literature. Figure [l] 
(page 12 1 provides the CFA that corresponds to the Algorithm [I] we analyze in 



our case study. 



Obtaining a process skeleton and a system instance from a CFA. Let us assume 
that SV , SVq, yl, r, n, RC, and N are given. Given a CFA A, we now define 
the process skeleton Sk(A) — {S, 5*°, R) induced by A. 

From the used variables and parameters we directly obtain the set S of states. 
We assume that all variables that range over D are initialized to 0. From this 
and SVq, we obtain 5°. 

It remains to define how the transition relation R is obtained from the seman- 
tics. For two relations i?i and i?2, we use the notation that Ri o R2 — {(x, z) \ 
{x,y) G i?i A (y, z) G R2}. Each path in the CFA A from qj to qp induces 
a sequence of operations uj = Oi, . . . , for some k; recall that the steps of a 
distributed algorithm are described by an acyclic CFA. Then |a;] is defined as 
|oi] o • • • o |ofc], and the transition relation is defined by i? = IJ^ p^^^j^ aI"^!- 

We have thus defined the process skeleton Sk(A) induced by CFA A. For a 
given p G PbCi a system instance lnst(p, Sk(A)) is then the parallel composition 
of iV(p) process skeletons Sk(yl), as defined in Section |3] 



5 Transferring Pseudo-code to our Framework 

We analyze Algorithm [T] which is the core of the broadcasting primitive 
by Srikanth and Toueg |3I]. In this section we first describe the computational 
model and Algorithm [l] from a distributed algorithms point of view, and will then 
show how to capture the algorithm in our modeling framework. 
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Algorithm 1 Core logic of the broadcasting algorithm from [52] , 

Code for processes i if it is correct: 

Variables 

1; Vi £ {false, true} 

2; accept^ G {false, true} <~ false 
Rules 

3; if Vi and not sent (echo) before then 
4; send (echo) to all; 

5; if received (echo) from at least t + 1 distinct processes 

and not sent (echo) before then 
6; send (echo) to all; 

7; if received (echo) from at least n — t distinct processes then 
8; accept^ <— TRUE; 



Computational model for asynchronous distributed algorithms. We recall the 
standard assumptions for asynchronous distributed algorithms. As mentioned in 
the introduction, a system consists of n processes out of which at most t may be 
faulty. When considering a fixed computation, we denote by / the actual number 
of faulty processes. It is assumed that n>3tAf<tAt>0. Correct processes 
follow the algorithm, in that they take steps that correspond to the algorithm 
description. Between every pair of processes, there is a bidirectional link over 
which messages are exchanged. A link contains two message buffers, each being 
the receive buffer of one of the incident processes. 

A step of a correct process is atomic and consists of the following three parts. 
First a process receives a possibly empty subset of the messages in its buffer, then 
it performs a state transition depending on its current state and the received 
messages. Finally, a process may send at most one message to each process, that 
is, it puts a message in the buffer of the other processes. 

Computations are asynchronous in that the steps can be arbitrarily inter- 
leaved, provided that each correct process takes an infinite number of steps. 
Moreover, if a message m is put into a process p's buffer, and p is correct, 
then m is eventually included in the set of messages received. This property is 
called reliable communication. Faulty processes are not restricted, except that 
they have no influence of the buffers of links to which they are not incident. This 
property is often called non-masquerading, as a faulty process cannot "pretend" 
to be another process. 

Specific details of Algorithm^ The code is typical pseudocode found in the dis- 
tributed algorithms literature. The lines [3}|8] describe one step of the algorithm. 
Receiving messages is implicit and performed before line [sj and the possible 
sending of messages is deferred to the end, and is performed after line [s] 

We observe that a process always sends to all. Moreover, lines [3}|8] only con- 
sider messages of type (echo), while all other messages are ignored. Hence, a 
Byzantine faulty process has an impact on correct processes only if they send an 
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^^^^ 



rcvd := e where rcvd < e A e < nsnt + / 
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[^(5^ = VI)) 



inc nsnt 



94 




-<9 



t + 1 < rcud I 
[sv = VO) 




96T [^(ct = VO)] 



1 < 



rcfd) j 



inc nsnt 



\n — t < rcvd 




98 



\ sv := AC I 




Fig. 1: Control flow automaton for the steps of Algorithm [l] on the left. On the 
right a CFA for a distributed algorithm tolerating clean crash faults. 



(echo) when they should not, or vice versa. Note that faulty processes may behave 
two-faced, that is, send messages only to a subset of the correct processes. More- 
over, faulty processes may send multiple (echo) messages to a correct process. 
However, from the code we observe that multiple receptions of such messages 
do not influence the number of messages received by "distinct" processes due 
to non-masquerading. Finally, the condition "not sent (echo) before" guarantees 
that each correct process sends (echo) at most once. 

Our modeling choices. The most immediate choice is that we consider the set 
of parameters 77 to be {rt,t, /} and RC{n,t, f) = n>3tAf<tAt>0. 
In the pseudo code, the status of a process is only implicitly mentioned. The 
relevant information we have to represent in the status variable is (i) the initial 
state (ii) whether a process has already sent (echo) and (iii) whether a process 
has set accept to true. Observe that once a process has sent (echo), its value 
of Vi does not interfere anymore with the further state transitions. Moreover, a 
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process only sets accept to true if it has sent a message (or is about to do so 
in the current step). Hence, we define the set 51^ to be {VO, VI, SE, AC}, where 
'S'^^o = {VO, VI}. VO corresponds to the case where initially Vi — false, and VI 
to the case where initially Vi — true. Further, SE means that a process has sent 
an (echo) message but has not set accept to true yet, and AC means that the 
process has set accept to true. Having fixed the status values, we can formalize 
the specifications we want to verify. They are obtained by the broadcasting 
specification parts called unforgeability, correctness, and relay introduced in |32j : 

G ([Vi. sv, 7^ VI] ^ G [Vj. svj ^ AC]) (U) 
G ([Vi. sv, = VI] ^ F [3j. svj ^ AC]) (C) 
G {[3i. sv, = AC] ^ F [Vj. svj = AC]) (R) 

Note carefully that ([u| is a safety specification while ([C]) and ([r]) are liveness 
specifications. 

As the asynchrony of steps is already handled by our parallel composition de- 
scribed in Section[3j it remains to describe the semantics of sending and receiving 
messages in our system model using control flow automata. 

Let us first focus on messages from and to correct processes. As we have 
observed that each correct process sends at most one message, and multiple 
messages from faulty processes have no influence, it would be sufficient to rep- 
resent each buffer by a single variable that represents whether a message of a 
certain kind has been put into the buffer. As we have only (echo) messages sent 
by correct processes, it is sufficient to model one variable per buffer. Moreover, 
if we only consider the buffers between correct processes, due to the "send to 
all" it is sufficient to capture all messages between correct processes in a single 
variable. To model this, we introduce the shared variable nsnt. 

The reception of messages can then be modeled by a local variable rcvd whose 
update depends on the messages sent. In particular, upon a receive, the variable 
rcvd can be increased to any value less than or equal to nsnt. 

It remains to model faults. As our system model is symmetric by construction, 
all processes must be identical processes. This allows at least the two possibilities 
to model faults: 

— we capture whether a process is correct or faulty as a fiag in the status, and 
require that in each run f <t processes are faulty. Then we would have to 
derive a CFA sub-automaton for faulty processes, and would need additional 
variables to capture sent messages by faulty processes. 

— we consider the system to consist of correct processes only, let N{n, t, /) = 
n — f, and model only the influence of faults, via the messages correct pro- 
cesses may receive. This can be done by allowing each correct process to 
receive at most / messages more than sent by correct ones, that is that rcvd 
can be increased to any value less than or equal to nsnt + /. 

Implementing the flrst option would require more variables, namely, the addi- 
tional flag to distinguish correct from faulty processes, and the additional vari- 
ables to capture messages by faulty processes. These variables would increase 
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the state space, and would make this option non-practical. Moreover, we would 
have to capture the number of faults /, and the corresponding resilience condi- 
tion. Therefore, we have implemented the latter approach for our experiments 
in Section [6l 

Based on this discussion we directly obtain the CFA given in Figure [l] that 
describes the steps of Algorithm [l] Note that its structure follows the pseudo 
code description of Algorithm [T] very closely. 

Verification strategy for liveness. Relevant liveness properties can typically only 
be guaranteed if the underlying system ensures some fairness guarantees. In asyn- 
chronous distributed systems one assumes for instance communication fairness, 
that is, every message sent is eventually received. The statement 3i. rcvdi < 
nsnti describes a global state where messages are still in transit. It follows that 
a formula ip defined by 

F G rcvdi < nsnti] (inequity) 

states that the system violates communication fairness. We only require a live- 
ness specification if to be satisfied if the system is communication fair. In other 
words, (fi is satisfied or the communication is unfair, that is, tpW'tl^. Our approach 
is to automatically verify (p\/ ip. 

Along all paths where communication is fair, the value of rcvdi has at least 
to reach the value of nsnti. Since rcvdi can only increase upon a step by i, i is 
forced to take steps as long as it has not received messages yet. That is, by this 
modeling, communication fairness implies some form of computation fairness. 



Modeling other fault scenarios. Fault scenarios other than Byzantine faults can 
be modeled by changing the system size, using conditions similar to (inequity), 
and slightly changing the CFA. More precisely, by changing the non-deterministic 
assignment (the edge leaving qj) that corresponds to receiving messages. For in- 
stance, replacing Byzantine by send omission process faults V2& could be modeled 
as follows: Faulty processes could be modeled explicitly by setting N{n, t, f ) = n. 
That at most / processes may fail to send messages, could be modeled by 
FG [3z. rcvdi + f < nsnti]. Finally, in this fault model processes may receive 
all messages sent, that is, rcvd := e where rcvd < s A s < nsnt. By similar 
adaptations one models, e.g., corrupted communication (e.g., due to value faulty 
links) [3D], or hybrid fault models [5] that contain different fault scenarios. 



6 Experimental Evaluation 

We have extended Spin's [17] input language Promela to be able to express 
our control flow automata that operate on unbounded variables and symbolic 
variables to express parameters. Figure [2] provides the central parts of the code 
of our case study. For the experiments we have implemented four distributed 
algorithms that use threshold guarded commands. They differ in the guarded 
commands, and work for different fault assumptions. The following list is ordered 



14 



symbolic int N, T, F; 

assume (N > 3 && F >= && T >= 1) ; 

assume (N > 3 * T && F <= T) ; 

atomic ex_acc = some(Proc:pc == AC); 
atomic all.acc = all(Proc:pc == AC); 

atomic in.transit = some (Proc : nrcvd < nsnt ) ; 

active [N - F] proctype ProcO {. 
byte pc = 0, next.pc = 0; 
int nrcvd = 0, next_nrcvd = 0; 
[. . .] 
do 

: : atomic { 
[. . .] 

if 

: : next_nrcvd >= N - T -> 

next_pc = AC ; 
: : next.nrcvd < N - T && 

(pc == VI II next.nrcvd >= T + 1) -> 

next_pc = SE ; 
: : else -> 

next_pc = pc ; 

fi; 

/* send the echo message */ 
if 

: : (pc == VO II pc == VI) && 

(next_pc == SE II next_pc == AC) -> 
nsnt ++ ; 
: : else ; 

[. . .] 

} 

od ; 

} 



Fig. 2: Example encoding of the CFA in our Promela extension. 



15 



^ parameter values 


spec valid 


Time 


Mem. 


Stored Transitions | Depth 


Byz 


Bl N=7,T=2,F=2 


( 


u 


\ / 


3.13 sec. 


74 MB 


193 • 10^ 


1 ■ 10" 


229 


B2 N=7,T=2,F=2 




c 


1 / 


3.43 sec. 


75 MB 


207 • 10" 


2- lO'' 


229 


B3 N=7,T=2,F=2 




R 


1 / 


6.3 sec. 


77 MB 


290 • 10" 


3- 10" 


229 


B4 N=7,T=3,F=2 




U 


] / 


4.38 sec. 


77 MB 


265 • 10" 


2 • 10" 


233 


B5 N=7,T=3,F=2 




c 


1 / 


4.5 sec. 


77 MB 


271 • 10" 


2 • 10" 


233 


B6 N=7,T=3,F=2 




R 


1 X 


0.02 sec. 


68 MB 


1 • 10" 


13 • 10" 


210 


OMIT 


Ol N=5,To=2,Fo=2 


( 


U 


\ / 


1.43 sec. 


69 MB 


51 • 10" 


878 • 10" 


175 


02 N=5,To=2,Fo=2 




c 


1 / 


1.64 sec. 


69 MB 


60 • 10" 


1 • 10" 


183 


03 N=5,To=2,Fo=2 




R 


1 / 


3.69 sec. 


71 MB 


92 • 10" 


2- 10" 


183 


04 N=5,To=2,Fo=3 




U 


] / 


1.39 sec. 


69 MB 


51 • 10" 


878 • 10" 


175 


05 N=5,To=2,Fo=3 




c 


1 X 


1.63 sec. 


69 MB 


53 • 10" 


1 ■ 10" 


183 


06 N=5,To=2,Fo=3 




R 


1 X 


0.01 sec. 


68 MB 


17 


135 


53 


SYMM 


SI N=5,T=l,Fp=l,Fs=0 


( 


U 


) / 


0.04 sec. 


68 MB 


3- 10" 


23 • 10" 


121 


S2 N=5,T=l,Fp=l,Fs=0 




c 


1 / 


0.03 sec. 


68 MB 


3- 10" 


24 • 10" 


121 


S3 N=5,T=l,Fp=l,Fs=0 




R 


1 / 


0.08 sec. 


68 MB 


5- 10" 


53 • 10" 


121 


S4 N=5,T=3,Fp=3,Fs=l 




U 


) / 


0.01 sec. 


68 MB 


66 


267 


62 


S5 N=5,T=3,Fp=3,Fs=l 




c 


1 X 


0.01 sec. 


68 MB 


62 


221 


66 


S6 N=5,T=3,Fp=3,Fs=l 




R 


1 / 


0.01 sec. 


68 MB 


62 


235 


62 


CLEAN 


CI N=3,Tc=2,Fc=2,Fnc=0 




U 


\ / 


0.01 sec. 


68 MB 


668 


7- 10" 


77 


C2 N=3,Tc=2,Fc=2,Fnc=0 




c 


1 / 


0.01 sec. 


68 MB 


892 


8- 10" 


81 


C3 N=3,Tc=2,Fc=2,Fnc=0 




R 


1 / 


0.02 sec. 


68 MB 


1 • 10" 


17 • 10" 


81 



Table 1: Summary of experiments 



from the most general fault model to the most restricted one. The given resilience 
conditions on n and t are the ones we expected from the literature, and their 
tightness was confirmed by our experiments: 

Byz. tolerates t Byzantine faults if n > 3t, 

SYMM. tolerates t symmetric (identical Byzantine [T) faults if n > 2t, 
OMIT, tolerates t send omission faults if n > 2t, 
CLEAN, tolerates t clean crash faults for n > t. 

The CFAs of these algorithms follow the same principles, so we do not give 
all of them in this paper. Figure [l] provides the most complicated one, namely 
Byz (we discussed how it is obtained from the literature in detail in Section [S]), 
next to the CFA of clean which actually is the simplest one. Our tool takes as 
input a CFA encoded in extended Promela, and concrete values for parameters, 
generates as output standard Promela code. 

The major goal of the experiments was to check the adequacy of our for- 
malization. To this end we considered the four mentioned well-understood dis- 



16 



unfotg, f=0 A 
unfotg, f=1 — A — 
unfotg, f=2 — ' — 
unfotg, f=3 — • — 
cott,f=o -B- 





4,1 4,2 4,3 5,1 5,2 5,3 6,1 6,2 6,3 7,1 7,2 7,3 8,1 8,2 8,3 9,1 



Fig. 3: Spin running time on BYZ. 



Fig. 4: Spin memory usage on BYZ. 



tributed algorithms. For each of which we systematicalljj^ changed the parameter 
values, in order to ascertain that under our modeling, the different combination 
of parameters lead to the expected result. Table [l] and Figures [3j [4] summarize 
the results of our experiments. 

Lines Bl - B3, 01 - 03, SI - S3, and CI - C3 capture the cases that are within 
the resilience condition known for the respective algorithm, and the algorithms 
were verified by Spin. In Lines B4-B6, the algorithm's parameters are chosen 
to achieve a goal that is known to be impossible [17]i i-6-i to tolerate that 3 out 
of 7 processes may fail. This violates the n > 3t requirement. Our experiment 
shows that even if only 2 faults occur in this setting, the relay specification (|r|) 
is violated. In Lines 04-06, the algorithm is designed properly, i.e., 2 out of 5 
processes may fail (n > 2t in the case of omission faults). Our experiments show 
that this algorithm fails in the presence of 3 faulty processes, i.e., ([C| and (|r|) 
are violated. 

For slightly bigger systems, that is, for n = 11 our experiments run out of 
memory. This shows the need for parameterized verification of these algorithms. 



7 Related Work 

In the area of verification, the most closest work to ours is [TB] which introduces 
a framework that targets at parameterized fault-tolerant distributed algorithms. 
However, [13] only considers fixed size (and finite) process descriptions which, 
consequently, cannot depend on the parameters n, t, and /. This makes it im- 
possible for the algorithms to use thresholds for two reasons: (i) With fixed 
size variables it is impossible to count messages in a parameterized setting, 
(ii) When process descriptions do not refer to parameters, it is impossible to 
compare counters against e.g., the parameter t, a standard construction in dis- 
tributed algorithms. While [13] contains ideas to model faults, the formalism 
does only allow to express very limited fault-tolerant distributed algorithms. For 
instance, their verification example considered a broadcasting algorithm in the 
case of crash faults, that has a trivial threshold guard, namely where one checks 

^ Complete experimental data is given in the appendix. 



17 



whether one message is received. As explained in the introduction, these kinds 
of rules are problematic in the presence of more severe fault types as Byzantine 
faults. Finally, the experimental data they provide is restricted to reporting that 
for f ~ 17 their algorithm was verified. The algorithm they considered as a case 
study works in a very simple setting, namely it is correct for all combinations 
of n and t, so they did not have to consider resilience conditions. However, fail- 
ure models that are more involved than their crash assumptions typically call 
for special constraints on n and t. Moreover, they use a specification of reliable 
broadcast which differs from the distributed algorithms literature (e.g., [TCI). In 
fact, their specification can be satisfied with a trivial algorithm (consisting of a 
single assignment Ey := -L). 

To conclude, [ISj considered the very important area of formal verification 
of fault-tolerant distributed algorithms, and showed what kind of modeling is 
feasible with techniques from regular model checking. From the points discussed 
above, it is clear the their approach although making interesting progress falls 
short of several aspects of fault-tolerant distributed algorithms. An important 
goal of our work is to initiate a systematic study of distributed algorithms from 
a verification and programming language point of view in a way that does not 
betray the fundamentals of distributed algorithms. We believe that ours is indeed 
the first paper about model checking of an adequately modeled fault-tolerant 
distributed algorithm. In our companion paper |18| , we even show how to verify 
the algorithms from the experiments above for all system sizes, and we thus 
actually verify the algorithms rather than just instances of the algorithms. 

The I/O Automata framework !24?19^25J models a distributed system as a 
collection of automata representing processes and of automata representing the 
communication medium, e.g., message passing links. The framework concentrates 
on the interfaces — input and output actions — rather than on semantics, and in 
fact much of the lOA literature on distributed algorithms uses pseudo code to 
describe what happens, e.g., upon an input event. In contrast to I/O Automata, 
which focus on the interfaces between processes (input, output), our CFAs fo- 
cus on the semantics for steps of distributed algorithms, and the construction 
of a system instance as a Kripke structure corresponds directly to standard dis- 
tributed computing models like [12l9ll0j , that are build around steps rather than 
input or output actions. 

The temporal logic of actions |22] is a variant of temporal logic by [28, , and 
built upon it, TLA+ is a specification language for concurrent and reactive sys- 
tems. This approach is very general, because one can express a wide variety of 
systems in TLA and TLA-I-. As our domain-specific framework is built specifi- 
cally for distributed algorithms, we focused on their specifics such as resilience 
conditions, faults, and asynchrony. 

8 Conclusions 

We introduced a framework to capture threshold-based fault-tolerant distributed 
algorithms. The framework consists of a parametric system model and of con- 
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trol flow automata, which allows to express the non-determinism typical for dis- 
tributed algorithms. We explained in detail how an algorithm from the literature 
can be formalized in this framework. 

We verified the appropriateness of our modeling by model checking four well- 
understood fault-tolerant distributed algorithms for fixed system sizes. This 
shows that the framework is a starting point to address many exciting verifi- 
cation problems in the area of distributed algorithms. In fact, in a companion 
paper |18j is dealing with the parameterized verification problem. There, we are 
using the framework to apply several abstraction techniques which allowed us 
to verify the four algorithms for all combinations of parameters admitted by the 
resilience condition. 
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APPENDIX 

A Experimental data 



This section provides a complete set of experiments. The highUghted Unes are 
the ones we have chosen for the body of the manuscript. 



# 


param 








spec valid 


SpinTime 


SpinMemory 


Stored Transitions 


Depth 


1 


N=3,T= 


l,Fs= 


l,Fp= 


=0 


unforg ■/ 


0.01 sec. 


68.019 MB 


533 3 • 10^ 


89 


2 


N=3,T= 


l,Fs= 


l,Fp= 


=0 


corr y 


0.01 sec. 


68.019 MB 


578 3 • 10^ 


89 


3 


N=3,T= 


l,Fs= 


l,Fp= 


:0 


relay / 


0.01 sec. 


68.019 MB 


826 6 • 10^ 


89 


4 


N=5,T= 


l,Fp= 


=0,Fs= 


=0 


unforg / 


0.6 sec. 


69.191 MB 


39 • 10^ 375 • 10^ 


177 


5 


N=5,T= 


l,Fp= 


=0,Fs= 


=0 


corr y 


0.62 sec. 


69.191 MB 


39 • 10^ 379 • 10^ 


177 


6 


N=5,T= 


l,Fp= 


=0,Fs= 


:0 


relay / 


1.56 sec. 


70.363 MB 


70 • 10^ 976 • 10^ 


177 



y 'N=5,T=l,i^'p=l,'Fs==6 dnforg/ 6.64 sec. %tm\m S-lo^ '^3' 

\ N=5,T=l,Fp=l,Fs=0 corr 7 0.03 sec. 68.019 MB 3-10^ 24 

> N.^5 J=4 g p=l.Fs=0 relay / 0.08 sec. _ _ 68-019 MB 5-10^ 53 

1^ 0.06 sec.""" 68.215 MB 5-10^ 45 

11 N=5,T=l,Fp^l,Fs=l corr ✓ 0.06 sec. 68.215 MB 6 • lO'^ 45 • 10^ 139 

12 N=5,T=l,Fp=l,Fs=l relay / 0.16 sec. 68.215 MB 10 • 10=* 111 • 10=* 139 

13 N=5,T=2,Fp=0,Fs=0 unforg/ 1.04 sec. 69.777 MB 53 ■ 10-* 515 ■ 10-* 188 

14 N=5,T=2,Fp=0,Fs=0 corr 7 0.89 sec. 69.777 MB 55 • lO'* 533 • 10=* l88 

15 N=5,T=2,Fp=0,Fs=0 relay / 1.81 sec. 70.754 MB 83 • 10=* 1 • lO'* 188 



16 N= 


=5,T= 


=2,Fp:. 


=l,Fs= 


=0 


unforg / 


0.04 sec. 


68.019 MB 


3 • 10=* 


29 • 10=* 


131 


17 N= 


:=5,T = 


=2,Fp= 


=l,Fs= 


=0 


corr 


0.05 sec. 


68.019 MB 


4 • 10=* 


33 • 10=* 


131 


18 N= 


=5,T= 


=2,Fp= 


=l,Fs= 


=0 


relay / 


0.08 sec. 


68.019 MB 


5-10=* 


49 • 10=* 


131 


19 N= 


=5,T= 


=2,Fp= 


=l,Fs= 


=1 


unforg / 


0.09 sec. 


68.215 MB 


7-10=* 


61 • 10=* 


149 


20 N= 


=5,T= 


=2,Fp= 


=l,Fs= 


=1 


corr y 


0.09 sec. 


68.215 MB 


8 • 10=* 


62 • 10=* 


149 


21 N= 


=5,T= 


=2.Fp= 


=LFs= 


=1 


re-lay ■/ 


0.18 s(k;. 


68.41 MB 


12 ■ 10=* 


125 ■ 10=* 


149 


22 N= 


=-kT= 


=2.Fp= 


=2.Fs= 


=0 


unforg / 


0.01 soc. 


()8.()19 MB 


32:-! 


1 ■ 10-'' 


8(i 


23 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=0 


corr ■/ 


0.01 sec. 


68.019 MB 


412 


2 • 10=* 


86 


24 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=0 


relay / 


0.01 sec. 


68.019 MB 


359 


2- 10=* 


88 


25 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=1 


unforg / 


0.01 sec. 


68.019 MB 


662 


3-10=* 


98 


26 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=1 


corr / 


0.01 sec. 


68.019 MB 


762 


4 • 10=* 


98 


27 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=1 


relay / 


0.01 sec. 


68.019 MB 


856 


6-10=* 


98 


28 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=2 


unforg / 


0.01 sec. 


68.019 MB 


1 • 10=* 


6-10=* 


110 


29 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=2 


corr / 


0.01 sec. 


68.019 MB 


1 • 10=* 


6 • 10=* 


110 


30 N= 


=5,T= 


=2,Fp= 


=2,Fs= 


=2 


relay / 


0.02 sec. 


68.019 MB 


1 • 10=* 


11 • 10=* 


110 


31 N= 


=5,T= 


=3,Fp= 


=0,Fs= 


=0 


unforg / 


1.01 sec. 


70.168 MB 


63 • 10=* 615 • 10=* 


199 


32 N= 


=5,T= 


=3,Fp= 


=0,Fs= 


=0 


corr / 


1.16 sec. 


70.363 MB 


69 • 10^ 


670 • 10^ 


199 


33 N= 


=5,T= 


=3,Fp= 


=0,Fs= 


=0 


relay ■/ 


1.61 sec. 


70.754 MB 


81 • 10=* 


967 • 10=* 


199 


34 N= 


=5,T= 


=3,Fp= 


=l,Fs= 


=0 


unforg / 


0.05 sec. 


68.019 MB 


4 • 10=* 


31 • 10=* 


141 


35 N= 


=5,T= 


=3,Fp= 


=l,Fs= 


=0 


corr / 


0.06 sec. 


68.019 MB 


4-10=* 


40 • 10=* 


141 


36 N= 


=5,T= 


=3,Fp= 


=l,Fs= 


=0 


relay / 


0.06 sec. 


68.019 MB 


4-10=* 


37 • 10=* 


143 
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37 N=5,T=3,Fp=l,Fs=l unforg / 0.11 sec. 68.215 MB 9 ■ 10^ 73 ■ lO"^ 159 

38 N=5,T=3,Fp=l,Fs=l corr ✓ 0.12 sec. 68.215 MB 10 ■ 10"^ 77 ■ 10^ 159 

39 N=5,T=3,Fp=l,Fs=l relay / 0.17 sec. 68.41 MB 12 • 10^ 112 • 10^ 159 

40 N=5,T=3,Fp^2,Fs^0 unforg / 0.01 sec. 68.019 MB 323 1 ■ lO^^ 90 

41 N=5,T=3,Fp=2,Fs=0 corr X 0.01 sec. 68.019 MB 296 l^W 94~ 

42 N=5,T=3,Fp=2,Fs=0 relay / 0.01 sec. 68.019 MB 322 1 ■ 10^ 90 

43 N=5,T=3,Fp=2,Fs=l unforg / 0.01 sec. 68.019 MB 710 lOCP 107 

44 N=5,T=3,Fp=2,Fs=l corr ✓ 0.01 sec. 68.019 MB 871 4 ■ IQ-^ 107 

45 N=5,T=3,Fp=2,Fs=l relay / 0.01 sec. 68.019 MB 763 4 • 10^ 109 

46 N^5,T=3,Fp^2,Fs=2 unforg ✓ 0.01 sec. 68.019 MB 1 ■ IQ-^ 7 ■ 10"^ 119 

47 N=5,T=3,Fp=2,Fs=2 corr 7 0.01 sec. 68.019 MB 1 • 10'^ 7 • 10'^ Tl9 

48 N=5,T=3,Fp=2,Fs=2 relay / 0.02 sec. 68.019 MB 1 • 10^ 10 • 10^ 119 

49 N=5,T=3,Fp=3,Fs=0 unforg / 0.01 sec. 68.019 MB 35 135 48 

50 N=5,T=3,Fp=3,Fs=0 corr X 0.01 sec. 68.019 MB 35 120 52 

51 N=5,T=3,Fp=3,Fs=0 relay / 0.01 sec. 68.019 MB 34 127 48~ 
12 N=5,T=3,Fp=3,Fs=l unforg / 0.01 sec. 68.019 MB 66 267 m 
B3 N=5,T=3,Fp=3,Fs=l corr X 0.01 sec. 68.019 MB 62 221 66" 



55 N=5,T=3,Fp=3,Fs=2 unforg / 0.01 sec. 68.019 MB 107 447 73 

56 N=5,T=3 Jp=3,Fs=2 corr / 0.01 sec. 68.019 MB 130 465 73 

57 N=5,T=3,Fp=3,Fs=2 relay / 0.01 sec. 68.019 MB 107 439 75 

58 N=5,T=3,Fp=3,Fs=3 unforg / 0.01 sec. 68.019 MB 148 635 79 

59 N=5,T=3,Fp=3,Fs=3 corr / 0.01 sec. 68.019 MB 166 597 79 

60 N=5,T^3,Fp^3,Fs=3 relay ✓ 0.01 sec. 68.019 MB 163 727 79 

61 N=ll,T^5,Fs^5,Fp:^0 unforg OOM 3.45e+03 sec. 3015.621 MB 59 • 10*^ 1.3010617e+09 1 • 10^ 

62 N=ll,T=5,Fs=5,Fp=0 corr OOM 3.46c+03 sec. 3015.816 MB 59 • 10** 1.3011555c+09 1 • 10^ 

63 N=ll,T=5,Fs=5,Fp=0 relay OOM 2.88e+03 sec. 3015.816 MB 59 • 10^ 1.1038422e+09 1 • 10^ 

Table 2: Symm 
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^ param 








spec 


valid 


SpinTime 


SpinMemory 


Stored 


Transitions 


Depth 


1 N= 


=2,Tc= 


=l,Fc= 


=l,Fnc= 


=0 


unforg 


/ 


0.01 sec. 


68.019 MB 


72 


533 


46 


2 N= 


=2,Tc= 


=l,Fc= 


=l,Fnc= 


=0 


corr 


/ 


0.01 sec. 


68.019 MB 


131 


863 


50 


3 N= 


=2,Tc= 


=l,Fc= 


=l,Fnc= 


=0 


relay 


/ 


0.01 sec. 


68.019 MB 


114 


1 • 


10-^ 


50 


4 N= 


=3,Tc= 


=l,Fc= 


=0,Fnc= 


=0 


unforg 


/ 


0.01 sec. 


68.019 MB 


578 


5- 


10^ 


83 


5 N= 


=3,Tc= 


=l,Fc= 


=0,Fnc= 


=0 


corr 


/ 


0.01 sec. 


68.019 MB 


759 


7- 


10=* 


83 


6 N= 


=3,Tc= 


=l,Fc= 


=0,Fnc= 


=0 


relay 


/ 


0.01 sec. 


68.019 MB 


829 


11 


• 10^ 


83 


7 N= 


=3,Tc= 


=l,Fc= 


=l,Fnc= 


=0 


unforg / 


0.01 sec. 


68.019 MB 


578 


5- 


10^ 


83 


8 N= 


=3,Tc= 


=l,Fc= 


=l,Fnc= 


=0 


corr 


/ 


0.01 sec. 


68.019 MB 


759 


7- 


10=* 


83 


9 N= 


=3,Tc= 


=l,Fc= 


=l,Fnc= 


=0 


relay 


/ 


0.01 sec. 


68.019 MB 


829 


11 


• 10^ 


83 


10 N= 


=3,Tc= 


=l,Fc= 


=l,Fnc= 


=1 


unforg 


/ 


0.01 sec. 


68.019 MB 


249 


2 • 


10^ 


84 


11 N= 


=3,Tc= 


=l,Fc= 


=l,Fnc= 


=1 


corr 


/ 


0.01 sec. 


68.019 MB 


424 


4- 


10^ 


76 


12 N= 


=3,Tc= 


=l,Fc= 


=l,Fnc= 


=1 


relay 


/ 


0.01 sec. 


68.019 MB 


332 


4- 


10=* 


77 


13 N= 


=3,Tc= 


=2,Fc= 


=0,Fnc= 


=0 


unforg 


/ 


0.01 sec. 


68.019 MB 


668 


7- 


10=* 


77 


14 N= 


=3,Tc= 


=2,Fc= 


=0,Fnc= 


=0 


corr 


/ 


0.02 sec. 


68.019 MB 


892 


8- 


10=* 


81 


15 N= 


=3,Tc= 


=2,Fc= 


=0,Fnc= 


=0 


relay 


/ 


0.02 sec. 


68.019 MB 


1 • 10^ 


17 


• 10=* 


81 


16 N= 


=3,Tc= 


=2,Fc= 


=l,Fnc= 


=0 


unforg 


/ 


0.01 sec. 


68.019 MB 


668 


7- 


10=* 


77 


17 N= 


=3,Tc= 


=2,Fc= 


=l,Fnc= 


=0 


corr 


✓ 


0.01 sec. 


68.019 MB 


892 


8- 


10=* 


81 


18 N= 


=3,Tc= 


=2,Fc= 


=l,Fnc= 


=0 


relay 


/ 


0.02 sec. 


68.019 MB 


1 • 10^ 


17 


• 10=* 


81 


19 N= 


=3,Tc= 


=2,Fc= 


=l,Fnc= 


=1 


unforg 


/ 


0.01 sec. 


68.019 MB 


279 


2- 


10=* 


77 


20 N= 


=3,Tc= 


=2,Fc= 


=l,Fnc= 


=1 


corr 


/ 


0.01 sec. 


68.019 MB 


425 


4- 


10=* 


78 


21 N= 


=3,Tc= 


=2,Fc= 


=l,Fnc= 


=1 


relay 


/ 


0.01 sec. 


68.019 MB 


475 


6- 


10=* 


78 


^2 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=0 


unforg 


/ 


0.01 sec. 


68.019 MB 


668 


7- 


10=* 


77 1 


23 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=0 


corr 


/ 


0.01 sec. 


68.019 MB 


892 


8- 


10=* 




g4 N= 


=3,Tc= 


=2.Fc= 


^Fnc= 


=0 








68.019 MB 


1-10^ 


17 


• 10=* 




25 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=1 


unforg / 


0.01 sec. 


68.019 MB 


279 


2 • 


10=^ 


77 


26 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=1 


corr 


/ 


0.01 sec. 


68.019 MB 


425 


4- 


10=* 


78 


27 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=1 


relay 


/ 


0.01 sec. 


68.019 MB 


475 


6- 


10=* 


78 


28 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=2 


unforg / 


0.01 sec. 


68.019 MB 


133 


1 ■ 


10^ 


72 


29 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=2 


corr 


/ 


0.01 sec. 


68.019 MB 


216 


2 • 


10=* 


71 


30 N= 


=3,Tc= 


=2,Fc= 


=2,Fnc= 


=2 


relay 


/ 


0.01 sec. 


68.019 MB 


198 


2- 


10=* 


72 


31 N= 


=3,Tc= 


=3,Fc= 


=0,Fnc= 


=0 


unforg 


X 


0.03 sec. 


68.019 MB 


561 


6- 


10=* 


78 


32 N= 


=3,Tc= 


=3,Fc= 


=0,Fnc= 


=0 


corr 


/ 


0.01 sec. 


68.019 MB 


1 • 10^ 


12 


• 10=* 


76 


33 N= 


=3,Tc= 


=3,Fc= 


=0,Fnc= 


=0 


relay 


/ 


0.04 sec. 


68.019 MB 


1 ■ 10-^ 


29 


■ 10=* 


76 


34 N= 


= 3,T(: = 


=3,F( = 


= l,Fiic= 


=0 


unforg 


X 


O.Ul sec. 


08.019 MB 


5()1 


G ■ 


lO'' 


7cS 


35 N= 


=3,Tc= 


=3,Fc= 


=l,Fnc= 


=0 


corr 


/ 


0.02 sec. 


68.019 MB 


1 • 10^ 


12 


• 10=* 


76 


36 N= 


=3,Tc= 


=3,Fc= 


=l,Fnc= 


=0 


relay 


/ 


0.04 sec. 


68.019 MB 


1 • 10^ 


29 


• 10=* 


76 


37 N= 


=3,Tc= 


=3,Fc= 


=l,Fnc= 


=1 


unforg 


/ 


0.01 sec. 


68.019 MB 


529 


5- 


10=* 


69 


38 N= 


=:3,TC = 


=3,Fc= 


=l,Fnc= 


=1 


corr 


/ 


0.01 sec. 


68.019 MB 


768 


7- 


10=* 


73 


39 N= 


=3,Tc= 


=3,Fc= 


=l,Fnc= 


=1 


relay 


X 


0.01 sec. 


68.019 MB 


92 


1 • 


10=* 


54 


40 N= 


=3,Tc= 


=3,Fc= 


=2,Fnc= 


=0 


unforg 


X 


0.01 sec. 


68.019 MB 


561 


6- 


10=* 


78 


41 N= 


=3,Tc= 


=3,Fc= 


=2,Fnc= 


=0 


corr 


/ 


0.02 sec. 


68.019 MB 


1 • 10^ 


12 


• 10=* 


76 


42 N= 


=3,Tc= 


=3,Fc= 


=2,Fnc= 


=0 


relay 


✓ 


0.04 sec. 


68.019 MB 


1 • 10^ 


29 


• 10=* 


76 
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43 N= 


=3,Tc= 


3,Fc=2,Fnc=l 


unforg 


/ 


0.01 sec. 


68.019 MB 


529 




5 • 10^ 


69 


44 N= 


=3,Tc= 


3,Fc=2,Fnc=l 


corr 


/ 


0.02 sec. 


68.019 MB 


768 




7-103 


73 


45 N: 


=3,Tc= 


3,Fc=2,Fnc=l 


relay 


X 


0.01 sec. 


68.019 MB 


92 




1 • 10^ 


54 


46 N 


=3,Tc= 


3,Fc=2,Fnc=2 


unforg 


/ 


0.01 sec. 


68.019 MB 


275 




2 • 10^ 


66 


47 N= 


=3,Tc= 


3,Fc=2,Fnc=2 


corr 


/ 


0.01 sec. 


68.019 MB 


446 




5 • 10^ 


70 


48 N: 


=3,Tc= 


3,Fc=2,Fnc=2 


relay 


/ 


0.01 sec. 


68.019 MB 


12 




86 


31 


49 N: 


=3,Tc= 


3,Fc=3,Fnc=0 


unforg 


/ 


0.01 sec. 


68.019 MB 


561 




6- 10^ 


78 


50 N 


=3,Tc= 


3,Fc=3,Fnc=0 


corr 


/ 


0.02 sec. 


68.019 MB 


1 • 10^ 


12 • 10^ 


76 


51 N= 


=3,Tc= 


3,Fc=3,Fnc=0 


relay 


/ 


0.03 sec. 


68.019 MB 


1 • 10^ 


29 • 10^ 


76 


52 N= 


==3,Tc= 


3,Fc=3,Fnc=l 


unforg 


/ 


0.01 sec. 


68.019 MB 


529 




5 • 10^ 


69 


53 N: 


=3,Tc= 


3,Fc=3,Fnc=l 


corr 


/ 


0.01 sec. 


68.019 MB 


768 




7-103 


73 


54 N 


=3,Tc= 


3,Fc=3,Fnc=l 


relay 


/ 


0.01 sec. 


68.019 MB 


92 




1 • 103 


54 


55 N= 


==3,Tc== 


3,Fc==3,Fnc=2 


unforg 


/ 


0.01 sec. 


68.019 MB 


275 




2 • 103 


66 


56 N= 


=3,Tc= 


3,Fc=3,Fnc=2 


corr 


/ 


0.01 sec. 


68.019 MB 


446 




5 • 103 


70 


57 N= 


=3,Tc= 


3,Fc=3,Fnc==2 


relay 


/ 


0.01 sec. 


68.019 MB 


12 




86 


31 


58 N: 


=3,Tc= 


3,Fc=3,Fnc=3 


unforg 


/ 


0.01 sec. 


68.019 MB 


238 




2 • 103 


49 


59 N= 


=3,Tc= 


3,Fc=3,Fnc=3 


corr 


X 


0.01 sec. 


68.019 MB 


249 




3-103 


53 


60 N 


=3,Tc== 


3,Fc=3,Fnc=3 


relay 




0.01 sec. 


68.019 MB 


12 




86 


31 


61 N: 


=ll,Tc 


=10,Fc==10,Fnc= 


:0 unforg 


OOM 6.75e+03 sec. 


3015.621 MB 59 • 


10*^ 


2.6021236e+09 757 


62 N: 


=ll,Tc 


=10,Fc=10,Fnc= 


=0 corr 


OOM 6.67e+03 sec. 


3015.621 MB 59 • 


10*^ 


2.6021237eH 


h09 761 


63 N 


=ll,Tc 


=10,Fc=10,Fnc= 


:0 relay 


OOM 6.72e+03 sec. 


3015.621 MB 59 • 


lO'^ 


2.6055872eH 


h09 761 



Table 3: Clean 
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param 






spec 


valid 


S p i nT i]Ti6 


S p i ri ]\^ Gmo ry 


stored 


Transitions 


Depth 


1 N= 


-4 T- 


=1 F= 


:1 


unforg 


;/ 


0.01 sec. 


68.019 MB 


533 


3-10^ 


82 


2 N= 


-4 T- 




:1 


corr 


/ 


0.01 sec. 


68.019 MB 


639 


3-10^ 


82 


3 N= 


-A T= 


:1 F = 


:1 


relay 


/ 


0.01 sec. 


68.019 MB 


706 


5- 10^ 


82 


4 N= 


:7.T = 


:1,F = 


:n 


uiiforp 


;/ 


278 sec. 


458.254 MB 


10 ■ 10" 


1.4322797O+08 319 


5 N= 


:7.T = 


:i.F = 


:() 


corr 


/ 


,'J2r) st^e. 


5i;i.;i:>2 MB 


li ■ iO" 


i.G.llGll.le+OH :il9 


6 N= 


■-7 T= 


:1,F = 


=0 


relay 


/ 


485 sec. 


603.371 MB 


14 • 10" 


2.4976492e+08 319 


7 N= 


■7 T- 




=1 


unforg / 


26 sec. 


114.504 MB 


1 • 10" 


14 • 10" 


268 


8 N= 


--7 T= 




:1 


corr 


/ 


31.2 sec. 


122.121 MB 


1 • 10" 


17 • 10" 


268 


9 N= 


--7,T= 


=1 F= 


:1 


relay 


/ 


44.7 sec. 


130.91 MB 


1 • 10" 


25 • 10" 


268 


10 N= 


■7 T- 

' 7 ^ 


:1,F= 


= 2 


unforg 


;X 


1.21 sec. 


70.558 MB 


74 • 10^ 


748 • 10^ 


241 


11 N= 


■7 T- 

' 7 ^ 


:1,F = 


:2 


corr 


X 


2.08 sec. 


72.316 MB 


127 ■ 10^ 


1 • 10" 


225 


12 N= 


-7 T- 


=1 F= 


=2 


relay 


X 


0.01 sec. 


68.019 MB 


42 


196 


222 


13 N= 


=7 T= 


:1 F= 


=3 


unforg X 


0.09 sec. 


68.215 MB 


7 • 10^ 


63 • 10^ 


201 


14 N= 


-7 T- 

' 7 ^ 


:1,F = 


=3 


corr 


X 


0.16 sec. 


68.41 MB 


13 • 10^ 


107 • 10^ 


181 


15 N= 


■7 T= 


=1 F= 

^7^ 


=3 


relay 


X 


0.01 sec. 


68.019 MB 


35 


127 


182 


16 N= 


=7 T= 


■2 F= 




unforg 


;/ 


416 sec. 


643.215 MB 


15 ■ 10" 


2.111452e+08 


325 


17 N= 


=7 T- 


:2,F = 


=0 


corr 


/ 


435 sec. 


665.48 MB 


15 ■ 10" 


2.1891381C+08 325 


18 N= 


-7 T- 


=2,F= 


=0 


relay 


/ 


859 sec. 


949.66 MB 


23 ■ 10" 


4.3596991C+08 325 


19 N= 


-7 T- 


2 F= 


:1 


unforg / 


38.1 sec. 


135.597 MB 


1 • 10" 


21 • 10" 


273 


20 N= 


■7 T= 


=2 F= 


:1 


corr 


/ 


40.3 sec. 


139.113 MB 


1 • 10" 


22 ■ 10" 


273 


21 N= 


=7 T= 

' 7 ^ 




:1 


relay 


/ 


77.9 sec. 


170.363 MB 


2 • 10" 


43 • 10" 


273 




-7 T- 


:2,F= 


:2 


unforg 


;/ 


'6.L'6 sec. 


T4'l)6 MB 


193 • 10'' 


1 ■ 10"' 






:7.T = 


=2,F= 


:2 


corr 


/ 


3.43 sec. 


75.051 MB 


207 ■ 10-' 


2- 10" 


229 




:7,T = 


:2,F = 


:2 


relay 


/ 


G.3 sec. 


77.98 MB 


290 ■ lO'' 


:i ■ 10'' 


229 Jj 


25 N= 


:7,T= 


=2,F= 


=3 


unforg 




0.11 sec. 


68.215 MB 


7 ■ 10'' 


60 ■ 10 ' 


205 


26 N= 


:7,T= 


=2,F= 


=3 


corr 


X 


0.21 sec. 


68.605 MB 


18 ■ 10-' 


143 ■ 10'' 


185 


27 N= 


:7,T= 


=2,F= 


=3 


relay 


X 


0.01 sec. 


68.019 MB 


33 


119 


176 


28 N= 


=7,T= 


=3,F= 


=0 


unforg 


;/ 


596 sec. 


876.418 MB 


21 ■ 10" 


2.967337e+08 


331 


29 N= 


=7,T= 


=3,F= 


=0 


corr 


/ 


604 sec. 


883.449 MB 


21 • 10" 


2.9891686e+08 331 


30 N= 


--7,T= 


=3,F= 


=0 


relay 


/ 


1.43e+03 sec. 


1678.902 MB 35 • 10" 


7.09111e+08 


331 


31 N= 


=7,T= 


=3,F= 


=1 


unforg / 


55.5 sec. 


162.551 MB 


2 ■ 10" 


29 ■ 10" 


278 


32 N= 


=7,T= 


=3,F= 


=1 


corr 


/ 


56 sec. 


163.722 MB 


2 • 10" 


30 • 10" 


278 


33 N= 


:7.T = 


=3,F= 


:1 


relay 


X 


0.83 sec. 


68.996 MB 


29 ■ 10'' 


461 ■ 10'' 


278 




=7,T= 


=3,F= 


-2 


unforg / 


4.38 sec. 


77.004 MB 


265 • 10^ 


2 • 10" 


233^ 


i5N= 


=7,T= 


=3,F_= 


■2 


corr 


/ 


4.5 sec. 


77.199 MB _ 


271 ■ 10'' 


2 ■ 10" 


23S 




37 N= 


:7,T= 


:3,F= 


-3 


unforg 


;/ 


0.32 sec. 


68.801 MB 


25 • 10'' 


209 • 10'' 


188 


38 N= 


:7,T= 


=3,F= 


=3 


corr 


/ 


0.33 sec. 


68.801 MB 


26 ■ 10^ 


215 • 10"* 


188 


39 N= 


=7,T= 


=3,F= 


=3 


relay 


X 


0.01 sec. 


68.019 MB 


100 


528 


165 



40 N=10,T=3,F=3 unforg OOM 2.02e+03 sec. 3015.621 MB 70 ■ 10" 9.9379608e+08 452 

41 N=10,T=3,F=3 corr OOM 2.12e+03 sec. 3015.816 MB 70 ■ 10" 9.9381042c+08 452 

42 N=10,T=3,F=3 relay OOM 2.97e+03 sec. 3015.816 MB 70 • 10" 1.4335274e+09 452 

Table 4: Byz 
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param 










SpiH.'^^imo Sp iri]\^Gmory 


stored 


Transitions 


Depth 


1 N= 


=3,To= 


l,Fo= 


1 


unforg 


/ 


0.01 sec. 68.019 MB 


440 




4-10" 


77 


2 N= 


:3,To= 


l,Fo= 


1 


corr 


/ 


01 sec 68 019 MB 


691 




7-10" 


85 


3 N= 


=3,To= 


l,Fo= 


1 


relay 


/ 


0.01 sec. 68.019 MB 


731 




10 ■ 10" 


85 


4 N= 


:5.To= 


l,Fo= 





imforg 


/ 


1.46 sec. 69.582 MB 


51 ■ 


10'^ 


878 ■ 10" 


175 


5 N= 


-'>.To= 


i,F(j= 





corr 


/ 


i. li sec. ()9.777 MB 


■52 ■ 


1()-' 891 ■ K)-' 


179 


6 N= 


=5,To= 


l,Fo= 





relay 


/ 


3.85 sec. 71.144 MB 


96 • 


10"* 


2 • lO'' 


179 


7 N= 


:5,To= 


l,Fo= 


1 


unforg 


/ 


1.38 sec. 69.582 MB 


51 • 


10^ 


878 • 10" 


175 


8 N= 


=5,To= 


l,Fo= 


1 


corr 


/ 


1 42 sec 69 777 MB 


53 • 


10"* 


909 • 10" 


183 


9 N= 


=5,To= 


l,Fo= 


1 


relay 


/ 


3.86 sec. 71.34 MB 


97- 


10'^ 


2 • lO'' 


183 


10 N= 


=5,To= 


l,Fo= 


2 


unforg 


/ 


1.38 sec. 69.582 MB 


51 • 


10^ 


878 • 10" 


175 


11 N= 


=5,To= 


l,Fo= 


2 


corr 


/ 


1.54 sec. 69.777 MB 


56 • 


10^ 


979 • 10" 


183 


12 N= 


=5,To= 


l,Fo= 


2 


relay 




01 sec 68 019 MB 


15 




131 


42 


13 N= 


=5,To= 


l,Fo= 


3 


unforg / 


1.39 sec. 69.582 MB 


51 • 


10^ 


878 • 10" 


175 


14 N= 


=5,To= 


l,Fo= 


3 


corr 


/ 


1.88 sec. 70.168 MB 


62 • 


10^ 


1 • lO'' 


183 


15 N= 


:5,To= 


l,Fo= 


3 


relay 


/ 


0.01 sec. 68.019 MB 


15 




131 


42 


16 N= 


=5,To= 


2,Fo= 





unforg / 


1 37 sec 69 582 MB 


51 • 


10^ 


878 • 10" 


175 


17 N= 


=5,To= 


2,Fo= 





corr 


/ 


1.49 sec. 69.972 MB 


57 • 


10^ 


945 • 10" 


179 


18 N= 


=5,To= 


2,Fo= 





relay 


/ 


3.56 sec. 70.949 MB 


88 ■ 


10'' 


2 ■ 10** 


179 


19 N= 


=5,To= 


2,Fo= 


1 


unforg 


/ 


1.38 sec. 69.582 MB 


51 ■ 


10^ 


878 ■ 10" 


175 


20 N= 


=5,To= 


2,Fo= 


1 


corr 


/ 


1.51 sec. 69.972 MB 


58 ■ 


10^ 


963 ■ 10" 


183 


21 N= 


=5,To= 


2,Fo= 


1 


relay 


/ 


3 53 sec 70 949 MB 


89 • 


10^ 


2 • lO** 


183 


22 N= 


=5,To= 


2,Fo= 


2 


unforg 


/ 


1.43 sec. 69.582 MB 


51 ■ 


lO"* 


878 • 10" 


175 


I3 N= 


z5,To= 


2,Fo= 


2 


corr 


/ 


1.64 sec. 69.972 MB 


60 ■ 


10" 


1 ■ 10" 


183 


M N= 


:5.T() = 


2,Fo= 


2 


rc'lay 


/ 


3.69 .sec. 71.144 MB 


92 ■ 


10-' 


2 ■ 10" 


183 


!5 N= 


:5,To= 


2,Fo= 


3 


unforg 


/ 


1.39 sec. 69.582 MB 


51 • 


lO" 


878 ■ 10" 


175 


!6 N= 


:5,To= 


2,Fo= 


3 


corr 


X 


1.63 sec. 69.777 MB 


53 ■ 


10" 


1 • 10" 


18^fl| 


i7 N= 


=5 To= 










sec^^^B^^^^^IH 






135 


5^1 


28 N= 


=5,To= 


3,Fo= 





unforg 


/ 


1.41 sec. 69.582 MB 


51- 




878 ■ 10" 


175 


29 N= 


=5,To= 


3,Fo= 





corr 


/ 


1.8 sec. 70.363 MB 


69- 


10" 


1 • 10** 


179 


30 N= 


:5,To= 


3,Fo= 





relay 


/ 


2.9 sec. 70.558 MB 


75- 


lO" 


1 • 10" 


179 


31 N= 


=5,To= 


3,Fo= 


1 


unforg / 


1.39 sec. 69.582 MB 


51- 


10" 


878 • 10" 


175 


32 N= 


=5,To= 


3,Fo= 


1 


corr 


/ 


1.83 sec. 70.363 MB 


70- 


10" 


1 • 10** 


183 


33 N= 


=5,To= 


3,Fo= 


1 


relay 


/ 


2.89 sec. 70.558 MB 


76- 


lO" 


1 • 10" 


183 


34 N= 


=5,To= 


3,Fo= 


2 


unforg / 


1.4 sec. 69.582 MB 


51 • 


10" 


878 • 10" 


175 


35 N= 


=5,To= 


3,Fo= 


2 


corr 


X 


1.37 sec. 69.582 MB 


47- 


10" 


851 • 10" 


183 


36 N= 


=5,To= 


3,Fo= 


2 


relay 




0.01 sec. 68.019 MB 


38 




257 


171 


37 N= 


=5,To= 


3,Fo= 


3 


unforg 


/ 


1.39 sec. 69.582 MB 


51 • 


10" 


878 • 10" 


175 


38 N= 


=5,To= 


3,Fo= 


3 


corr 


X 


1.65 sec. 69.777 MB 


53- 


10" 


1 • 10" 


183 


39 N= 


=5,To= 


3,Fo= 


3 


relay 


X 


0.01 sec. 68.019 MB 


38 




257 


171 


40 N= 


=ll,To 


=5,Fo 


=5 unforg OOM 6.97e+03 sec. 2757.347 MB 59 • 


10** 


2.60E+009 


757 


41 N= 


= ll,To 


=5,Fo 


=5 


corr 


OOM 7.25C+03 sec. 3015.621 MB 59 ■ 


10^ 


2.7891908cH 


-09 765 


42 N= 


=ll,To 


=5,Fo 


=5 relay 


OOM 9.82e+03 sec. 3015.621 MB 59 • 


lO** 


3.7808961e+09 765 



Table 5: Omit 
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